User Equipment Research Articles

Optimization of Energy Efficiency for Federated Learning over Unmanned Aerial Vehicle Communication Networks

Federated learning (FL) is considered a promising machine learning technique that has attracted increasing attention in recent years. Instead of centralizing data in one location for training a global model, FL allows the model training to occur on user devices, such as smartphones, IoT devices, or local servers, thereby respecting data privacy and security. However, implementing FL in wireless communication faces a significant challenge due to the inherent unpredictability and constant fluctuations in channel characteristics. A key challenge in implementing FL over wireless communication lies in optimizing energy efficiency. This holds significant importance, especially considering user devices with restricted power resources. On the other hand, unmanned aerial vehicle (UAV) technologies present a cost-effective solution owing to flexibility and mobility compared to terrestrial base stations. Consequently, the deployment of UAV communication in FL is viewed as a potential approach to deal with the energy efficiency challenge. In this paper, we address the problem of minimizing the total energy consumption of all user equipment (UE) during the training phase of FL over a UAV communication network. Our proposed system facilitates UE to operate concurrently at the same time and frequency, thereby improving bandwidth utilization efficiently. In this paper, we address the problem of minimizing the total energy consumption during the training phase of FL over a UAV communication network. To deal with the proposed nonconvex problem, we propose a novel alternating optimization approach by dividing the problem into two suboptimal problems. We then develop iterative algorithms based on the inner approximation method, yielding at least one locally optimal solution. The numerical results demonstrate the superiority of the proposed algorithm in solving the proposed problem compared to other benchmark algorithms, particularly in determining the optimal trajectory of the UAVs. In addition, we conduct extensive experiments to evaluate how different parameter settings affect performance after implementing the proposed optimization approaches for deploying FL within the UAV communication system. These analyses yield valuable insights into the comparative effectiveness of the proposed optimization algorithms concerning overall energy consumption reduction.

A Novel RSS Fingerprint to Locate User Equipment UE in Remote Area

Location determination algorithms are widely used in cellular networks, especially in the long-term evolution (LTE) network, to enable the provision of location-based services (LBS). The increasing global demand for cellular networks has resulted in the creation of new user equipment (UE) positioning systems that align with the network's momentum. Regrettably, all of these technologies are hindered by their incapacity to ascertain the location of the UE in distant regions. This article introduces a novel method utilizing Radio Frequency (RF) fingerprinting to precisely locate UEs in remote areas. The approach entails employing a proposed partitioning model with a high level of precision, incorporating artificial intelligence and machine learning AI/ML in its fundamental state to reduce the search area. Furthermore, two algorithms are suggested: The first aims to enhance the efficiency of the battery with limited capacity by decreasing the frequency of measurements transmission. The second utilizes Jaccard similarity and incorporates the prefix filtering technique to determine matches. The algorithm is used to speed up the process of matching the fingerprint recorded in the fingerprint database with the fingerprint captured in real time. The results shows that it can reduces the transmission rate by 77.08% and achieves the lowest error rate of 35.34 m. Additionally, it exhibits a response time of 8 seconds.

Deep Reinforcement Learning techniques for dynamic task offloading in the 5G edge-cloud continuum

The integration of new Internet of Things (IoT) applications and services heavily relies on task offloading to external devices due to the constrained computing and battery resources of IoT devices. Up to now, Cloud Computing (CC) paradigm has been a good approach for tasks where latency is not critical, but it is not useful when latency matters, so Multi-access Edge Computing (MEC) can be of use. In this work, we propose a distributed Deep Reinforcement Learning (DRL) tool to optimize the binary task offloading decision, this is, the independent decision of where to execute each computing task, depending on many factors. The optimization goal in this work is to maximize the Quality-of-Experience (QoE) when performing tasks, which is defined as a metric related to the battery level of the UE, but subject to satisfying tasks’ latency requirements. This distributed DRL approach, specifically an Actor-Critic (AC) algorithm running on each User Equipment (UE), is evaluated through the simulation of two distinct scenarios and outperforms other analyzed baselines in terms of QoE values and/or energy consumption in dynamic environments, also demonstrating that decisions need to be adapted to the environment’s evolution.

Multi-agent deep reinforcement learning for trajectory planning in UAVs-assisted mobile edge computing with heterogeneous requirements

In heterogeneous wireless networks, massive user equipments (UEs) generate computing tasks with time-varying heterogeneous requirements. To improve the service quality, this paper formulates a unmanned aerial vehicles (UAVs)-assisted mobile edge computing (MEC) framework for time-varying heterogeneous task requirements. In the framework, the task delay and the number of successfully executed tasks are optimized by jointly controlling the trajectories of multiple UAVs. To address the considered trajectory planning optimization problem, a collaborative multi-agent deep reinforcement learning (MADRL) algorithm is proposed, where each UAV is regarded as a learning agent. First, a counterfactual inference based personalized policy update mechanism is proposed to evaluate the independent policy of agents by comparing the policy with a designed counterfactual policy. Based on this idea, each agent updates a personalized policy from both group and individual interests to improve its cooperation ability in dynamic and complex environments. Then, a diversified experience sampling mechanism is proposed to enhance the efficiency of policy evaluation and update with rich experiences provided by the environment interaction and the modified whale optimization algorithm. Finally, evaluation results demonstrate the superiority and effectiveness of the proposed MADRL algorithm.

A decentralized HC-ADMM approach for large antenna arrays

This work addresses the evolving landscape of internet of things (IoT) applications and large antenna array systems, where optimizing spectral efficiency and simplifying design complexities are crucial. Focusing on two key challenges, the study introduces a novel hybrid analog-digital transceiver strategy tailored for frequency-selective channels. By integrating Shannon and Hartley theorems, the approach enhances data transfer rates, thereby optimizing radio frequency (RF) chain utilization in large-scale antennas. To achieve a balance between transceiver performance and hardware complexity, the study employs a decentralized alternating direction method of multipliers (ADMM) framework. The proposed hybrid consensus ADMM algorithm (HC-ADMM) ensures efficient convergence in decentralized optimization scenarios. Comparative analyses with ADMM and existing system transceiver optimization (ESTO) models highlight HC-ADMM's superior performance across key metrics such as spectral efficiency, efficiency per cell, total efficiency, and optimal scheduling of user equipment (UEs). Particularly notable is HC-ADMM's advanced optimization capabilities as the number of transmit antennas increases, positioning it as a promising approach for enhancing overall communication network performance.

Intelligent Dynamic Power Control with Cell Range Expansion for Small-Cells in 5G HetNets

In 5G heterogeneous networks (HetNets), small-cell base stations (SBSs) are deployed in the coverage of macro base stations (MBSs) to improve the system performance. However, some macro user equipments (MUEs) have strong interference from neighboring SBSs and thus the performance of MBSs decreases. Thus, in this paper, we propose a novel intelligent dynamic power control (DPC) with cell range expansion (CRE) to improve the downlink performance of both small-cell user equipments (SUEs) and CRE user equipments (CUEs) in 5G HetNets. That is, in the proposed DPC scheme, each MUE first collects the received signal strength indicator (RSSI) measurements from neighboring SBSs and sends them to the serving MBS. Then, the MBS finds MUEs with strong interference from neighboring SBSs based on a given target threshold of CRE and offloads a fraction of MUEs from MBSs to SBSs. In addition, SBSs divide their SUEs and CUEs into two groups, i.e., inner and outer groups, to assign different subchannels and dynamically allocate the appropriate transmission power to increase the performance of both SUEs and CUEs. Through simulation results, it is shown that the proposed DPC scheme outperforms others in terms of the capacity and outage probability of SUEs and CUEs.

Minimizing Task Age upon Decision for Low-Latency MEC Networks Task Offloading with Action-Masked Deep Reinforcement Learning.

In this paper, we consider a low-latency Mobile Edge Computing (MEC) network where multiple User Equipment (UE) wirelessly reports to a decision-making edge server. At the same time, the transmissions are operated with Finite Blocklength (FBL) codes to achieve low-latency transmission. We introduce the task of Age upon Decision (AuD) aimed at the timeliness of tasks used for decision-making, which highlights the timeliness of the information at decision-making moments. For the case in which dynamic task generation and random fading channels are considered, we provide a task AuD minimization design by jointly selecting UE and allocating blocklength. In particular, to solve the task AuD minimization problem, we transform the optimization problem to a Markov Decision Process problem and propose an Error Probability-Controlled Action-Masked Proximal Policy Optimization (EMPPO) algorithm. Via simulation, we show that the proposed design achieves a lower AuD than baseline methods across various network conditions, especially in scenarios with significant channel Signal-to-Noise Ratio (SNR) differences and low average SNR, which shows the robustness of EMPPO and its potential for real-time applications.

Investigation of Flow and Combustion Technology Changes in a Device Nozzle for Natural Gas as a Result of Mixing in Hydrogen

The efforts of the European Union (2021, EU Regulation on Gas and Hydrogen Networks) have been waiting for a solution to the problem of -even if only partial- independence from natural gas. The article shows the effects on the combustion technical parameters and equipment, if the 100% natural gas plant is replaced with a certain proportion of hydrogen. The article examines how it affects the combustion technical parameters of natural gas user equipment if the gas that feeds them is replaced by hydrogen-natural gas mixtures of different proportions. The results show that with the introduction of hydrogen, the Wobbe number decreases, and with this equipment, hydrogen can be added to the natural gas in an amount of 23 V/V% so that it meets the standards. When examining the flame propagation speed, we conclude that it is expected that the flame will not burn back at this ratio. The work promotes the progress of natural gas-hydrogen co-combustion technologies, as it examines a special part of the combustion equipment, the nozzle, and the flow engineering processes taking place in it. With the help of this model, we successfully investigated the excess air resulting from various natural gas-hydrogen mixtures in the case of a given geometry, but the model, with its mathematical additions, also proves that mixing in hydrogen generally reduces the amount of incoming air.

Joint computational and optimized resource allocation in edge computing networks

Fog computing has several undeniable benefits, such as enhancing near-real-time response, reducing transmission costs, and facilitating IoT analysis. This technology is poised to have a significant impact on businesses, organizations, and our daily lives. However, mobile user equipment struggles to handle the complex computing tasks associated with modern applications due to its limited processing power and battery life. Edge computing has emerged as a solution to this problem by relocating processing to nodes at the network’s periphery, which have more computational capacity. With the rapid evolution of wireless technologies and infrastructure, edge computing has become increasingly popular. Nevertheless, managing fog computing resources remains challenging due to resource constraints, heterogeneity, and distant nodes. For delay-sensitive intelligent IoT applications within the fog computing architecture, cooperation and communication processing resources in 6 G and future networks are essential. This study proposes a joint computational and optimized resource allocation (JCORA) technique to accelerate the processing of data from intelligent IoT sensors in a cell association environment. The proposed technique utilizes an uplink and downlink power allocation factor and the shortest job first (SJF) task scheduling system to optimize user fairness and decrease data processing time. This is a complex assignment due to several non-convex limitations. The suggested JCORA-SJF model simultaneously optimizes time partitioning, computing task processing mode selection, and target sensing location selection to maximize the weighted total of task processing and communication performance. The simulation results demonstrate the effectiveness of the proposed JCORA-SJF algorithms, and the system’s scalability is also examined.

Low-Complexity User Scheduling Framework for mmWave Hybrid Beam-forming OFDM System under Non-Ideal CSI

Introduction: User scheduling in millimeter-wave (mmWave) multi-user hybrid beam-forming Orthogonal Frequency Division Multiplexing (OFDM) systems involves the joint optimization of resource block group (RBG) allocation, beam pairing, and user selection. However, choosing the optimal scheduled User Equipment (UE), allocated RBGs, and communicating beams in practical mmWave hybrid beam-forming systems with non-ideal Channel State Information (CSI) remains challenging. Method: In this paper, we propose a low-complexity user scheduling framework. On one hand, two user classification methods are proposed under non-ideal CSI, assisted by beam search and RBG allocation respectively. On the other hand, in order to ensure both the throughput and fairness performance, a novel user selection scheme, called weighted user selection based on feedback threshold (WUSFT) scheme is proposed, and approximate closed-form expression of fairness index for both full feedback and feedback based on threshold are derived. Our proposed methods consist of three steps. First, RBGs are allocated based on the maximum received signal power (MRSP) of each user on each RBG, utilizing quasi-omnidirectional beams. In the second step, the communicating beams are further searched to achieve the MRSP with the allocated RBG. Result: Finally, the allocated RBG, or the determined beam information obtained through the beam search, is used to represent the correlation of user channels and classify the UEs into groups. Only UEs whose reported factor is not less than the feedback threshold will report received information. This simplifies the operation of user scheduling. Moreover, simulation results demonstrate that our proposed schemes can achieve more than 92.3% of the sum rate performance of exhaustive search methods while maintaining relatively low complexity. Conclusion: In addition, the user feedback overhead (FO) reduces obviously, especially with large UE number.

A 3.2-3.8 GHz low-noise amplifier for 5G/6G satellite-cellular convergence applications

The rapid evolution of wireless communication systems towards 5G standards has imposed stringent requirements on the performance of radio frequency front-end components. Among these, the Low-Noise Amplifier (LNA) plays a pivotal role in determining the overall noise figure and sensitivity of the receiver chain. This paper presents a comprehensive design and analysis of a 3.2-3.8 GHz LNA tailored for 5G applications, employing a 0.3 μmm gate length Gallium Arsenide (GaAs) pseudomorphic high electron transistor (pHEMT) technology process. The proposed LNA design focuses on achieving a low noise figure (NF), high gain, and robust linearity to accommodate the dense signal environment and wide bandwidth of 5G networks. The design leverages advanced matching networks and feedback topologies to enhance stability and reduce the noise contribution from the active devices. Simulation results predict a noise figure of 1.3-1.4 dB, a gain of 20-21 dB across the band of interest, and an input-referred third-order intercept point (IIP3) of 18 dBm. The LNA demonstrates excellent performance in a 5G testbed, showing a significant improvement in the signal-to-noise ratio and the potential to enhance 5G receiver sensitivity. The research substantiates the LNA’s viability for integration into 5G base stations and user equipment, underscoring its potential to contribute to the efficient and reliable operation of next-generation wireless networks. This LNA can be used for 5G New Release (NR) band of n77 and n78 (3.5-3.7 GHz)

5G new radio multiuser precoding in geostationary satellite networks

SummaryThe multiuser precoding procedures standardized by the 3rd Generation Partnership Project (3GPP) for new radio (NR) represent a new opportunity to realize satellite precoding for capacity improvement, given the integration of non‐terrestrial networks in the 3GPP framework. This work analyses and numerically evaluates them for the geostationary satellite scenarios consolidated by the 3GPP. It shows that the preferred precoding matrix indicated by NR codebooks is not able to handle properly the inter‐beam interference and should only be used to estimate the propagation channel. To this aim, we propose the use of modified type II port selection codebooks reporting on the strongest beams, which correspond to the higher interferences. Since the obtention of high precoding gains requires the operation in rather high signal‐to‐noise regimes and the proper estimation of the interfering contributions from many beams, we propose to enlarge the maximum number of beams reported by each precoding matrix indicator (PMI) report, currently limited to four. As alternative, we also evaluate a sequential multi‐report framework considering the effects of channel‐state information aging and the user equipment (UE) processing times. It preserves the current limits on the number of reported beams but increases the system overhead signalling. The quantization of estimated coefficients is also analysed, not only from the overhead point of view but also to properly report weak interferers corresponding to distant beams, due to the increase in the number of reported beams. Moreover, we identify the minimum interference power a UE should be able to detect to achieve the theoretical large precoding gains.

Joint beamforming and power splitting design for MISO downlink communication with SWIPT: a comparison between cell-free massive MIMO and small-cell deployments

Simultaneous wireless information and power transfer (SWIPT) has been advocated as a highly promising technology for enhancing the capabilities of 5G and 6G devices. However, the challenge of dealing with large propagation path loss poses a significant hurdle. To address this issue, massive multiple-input multiple-output (MIMO) is employed to enhance the efficiency of SWIPT in cellular-based networks with multiple small cells, and especially increase the energy for cell-edge users. In addition, by leveraging a large set of spatially distributed base stations to collaboratively serve SWIPT-enabled user equipment, the cell-free massive MIMO has the potential to provide even better performance than the conventional small-cell systems. In this work, we extend the investigation to include the application of SWIPT technology with alternating current (AC) logic in the cell-free networks and the small-cell networks and propose joint beamforming and power splitting optimization frameworks to maximize the system sum-rate, subject to the constraints on harvested energy, AC logic energy supply, and total transmit power. The optimization problem is shown to be non-convex, posing a significant challenge. To address this challenge, we resort to a two-stage decomposition approach. Specifically, we first introduce quadratic transform-based fractional programming (FP) algorithms to iteratively solve the non-convex optimization problems in the first stage, achieving near-optimal solutions with low time complexities. To further reduce the complexities, we also incorporate conventional schemes such as zero forcing, maximum ratio transmission, and signal-to-leakage-and-noise ratio for the design of beamforming vectors. Second, to determine the optimal power splitting ratio within the framework, we develop a one-dimensional (1-D) search algorithm to tackle the single variable optimization problem reduced in the second stage. These algorithms are then evaluated in the context of cell-free MIMO and small-cell networks with numerical experiments. The results show that the FP-based algorithms can consistently outperform those utilizing the conventional beamforming schemes, and the solutions of this work can achieve up to fivefold improvement in the system sum-rate than the small-cell counterpart while providing different but comparable performance trends in energy harvesting (EH).

D2D Self Organization in IOT via Triple Modular Redundancy Based MDS Code

Device-to-Device (D2D) communication is a key enabling technology to replace base station-to-device(B2D) communication. D2D with IoT realization has spectral and energy-efficient features for on-time quality data transfer. However, the rapid growth of such networks and the dynamic quality of service demands of user equipment require robust and efficient service quality in disasters, outages, and adversarial conditions. To address this challenge, this paper proposes a triple modular redundancy based on the maximum distance separable (TMR-MDS) technique has been proposed, which modifies the repair bandwidth by utilizing more storage occupation and improving the D2D link availability in the system. The proposed TMR-MDS technique will be used, which is composed of two phases: trust management modules and triple modular redundancy storage modules. As part of the initial stage of trust management, a fuzzy logic model is employed to determine the user’s trust value after the user’s behaviour is examined. Second, using the majority voting scheme to improve triple modular redundancy storage. Furthermore, if any problem occurs in the node, then it will self-organize. MATLAB is used to evaluate the performance of the proposed system. The overall system performance is evaluated in terms of the network’s trust level, and the communication cost of the proposed strategy. As per the experimental results, the proposed TMR-MDS method has a better performance of 12.67%, 23.68%, and 25.87% than hierarchical bi-partite, double replication MDS, and unequal repairing locality code.

Joint optimization of phase shift and task offloading for RIS-assisted multi-access edge computing in beyond 6G communication

Beyond 6G services and applications demand high and efficient processing capacity due to the massive connectivity of users equipment (UEs). However, the high computational capability and energy consumption of UEs are limited, which becomes a main challenge to overcome. Multi-access edge computing (MEC) has recently been studied widely as it can potentially assist complex tasks executed at UEs. Furthermore, several techniques have been proposed to optimize task offloading among users. Thus, another challenge in MEC is emerging due to the fact that mobile users do not always have a line-of-sight (LoS) to the base station (BS) due to the blocking object. Therefore, it can affect users data rate and result in incremental energy consumption. This research introduces the concept of reconfigurable intelligence surfaces (RIS) to support multiple-input-single-output (MISO) base stations (BS) in both uplink (UL) and downlink (DL) using BCD algorithms. While previous studies concentrate on enhancing task offloading and neglecting inter-user interference, this study suggests an optimization approach for UL and DL data rates, as well as minimizing task offloading delays. The results indicate that optimizing task placement, phase shift, and precoding can reduce the duration of task offloading.

The Physical Cell Identity Assignment Problem: A Practical Optimization Approach

The Physical Cell Identity (PCI) is a critical parameter to radio access networks used by the user equipment, like a smartphone, to identify the cell to connect. The PCI misassignment leads to call drops, no voice/data connections, and other sorts of impairments. Many works in the literature only treat the PCI assignment problem from the technological constraints viewpoint. In this work, we take a realistic approach and incorporate several operational constraints found daily in carrier-grade networks. For that, we model the problem with multiple objectives with a strict lexicographical dominance relation. We solve it using a hybrid memetic algorithm with warm start solutions obtained by a constraint programming solver and intensification with path-relinking and custom local search procedures. The experimental results suggest that our approach is superior to commercial solver packages in several scenarios.

Performance evaluation of pair selection algorithms in device-to-device communication using relay-assisted techniques

Communication services that are dependable are crucial, particularly during emergencies when the regular infrastructure for communication may be disrupted or nonexistent. In such situations, device-to-device (D2D) communication can be a helpful choice since it allows user equipment (UE) that is close to one another to connect directly, bypassing the cellular network infrastructure. The primary focus of this thesis is the application of D2D communication in a decentralized emergency scenario with a damaged eNodeB. The main objective is to find an appropriate strategy for finding and selecting D2D couples by simulating several methods in MATLAB. This study compares three D2D pair selection algorithms: distance-based, Signal-to-Interference and Noise Ratio (SINR)-based, and data rate-based distance-based. The simulation results show that the data rate-based strategy is the most effective method for selecting D2D couples in emergency scenarios. In contrast to algorithms that rely on distance and SINR, this one reduces the chance of an outage by 20%. Bit error rate (BER), capacity, spectral efficiency, and energy efficiency are the three types of links that are assessed: direct links, relay links, and UE relay links. The results show that, with the lowest BER and maximum data throughput, the direct link is the most reliable and efficient communication option. However, the relay connection and the UE relay link show better overall spectral efficiency in comparison to the direct link, indicating their ability to transport more data per unit of bandwidth. The option that consumes the least energy among the three is the direct link. The study demonstrates the great potential of D2D communication in emergency scenarios where conventional communication infrastructure may not be available. The direct link is the most dependable and effective alternative for communication, according to the data, although the UE link can still function effectively in the event that the direct link is compromised. The data rate-based method is a useful strategy for finding and choosing D2D partners. The results of this study can direct the development of D2D emergency communication solutions in 5G networks.

Random Access Preamble Design and Timing Advance Estimation for OTFS Systems in High-Mobility Scenarios

The uplink timing synchronization is indispensable for establishing a reliable link between the base station and the user equipment (UE). To tackle this problem, a new random access preamble (RAP) waveform is designed for cellular communication systems operating in high-mobility scenarios for orthogonal time frequency space systems. Meanwhile, we also propose a scheme to detect access UEs and estimate the corresponding round-trip propagation delay (i.e., timing advance (TA)). Concretely, the RAP root sequence consists of chirp sequences in the delay-Doppler domain, and the set of available RAPs is generated by cyclic shifting the root sequence. Then, the cross-correlation in the fractional Fourier domain is carried out to achieve the RAP detection and TA estimation of the access UEs. Exhaustive numerical simulation results corroborate that the proposed scheme has a superior RAP detection and TA estimation performance and exhibits sufficient robustness regarding Doppler shifts and the number of UEs.

Multi-IRS-Assisted mmWave UAV-BS Network for Coverage Extension.

In the era of Industry 5.0, advanced technologies like artificial intelligence (AI), robotics, big data, and the Internet of Things (IoT) offer promising avenues for economic growth and solutions to societal challenges. Digital twin technology is important for real-time three-dimensional space reproduction in this transition, and unmanned aerial vehicles (UAVs) can support it. While recent studies have explored the potential applications of UAVs in nonterrestrial networks (NTNs), bandwidth limitations have restricted their utility. This paper addresses these constraints by integrating millimeter wave (mmWave) technology into UAV networks for high-definition video transmission. Specifically, we focus on coordinating intelligent reflective surfaces (IRSs) and UAV networks to extend coverage while maintaining virtual line-of-sight (LoS) conditions essential for mmWave communication. We present a novel approach for integrating IRS into Beyond 5G/6G networks to enhance high-speed communication coverage. Our proposed IRS selection method ensures optimal communication paths between UAVs and user equipment (UE). We perform numerical analysis in a realistically modeled 3D urban environment to validate our approach. Our results demonstrate significant improvements in the received SNR for multiple UEs upon the introduction of IRSs, and they confirm the feasibility of coverage extension in mmWave UAV networks.

Optimized Two-Tier Caching with Hybrid Millimeter-Wave and Microwave Communications for 6G Networks

Data caching is a promising technique to alleviate the data traffic burden from the backhaul and minimize data access delay. However, the cache capacity constraint poses a significant challenge to obtaining content through the cache resource that degrades the caching performance. In this paper, we propose a novel two-tier caching mechanism for data caching on mobile user equipment (UE) and the small base station (SBS) level in ultra-dense 6G heterogeneous networks for reducing data access failure via cache resources. The two-tier caching enables users to retrieve their desired content from cache resources through device-to-device (D2D) communications from neighboring users or the serving SBS. The cache-enabled UE exploits millimeter-wave (mmWave)-based D2D communications, utilizing line-of-sight (LoS) links for high-speed data transmission to content-demanding mobile UE within a limited connection time. In the event of D2D communication failures, a dual-mode hybrid system, combining mmWave and microwave μWave technologies, is utilized to ensure effective data transmission between the SBS and UE to fulfill users’ data demands. In the proposed framework. the data transmission speed is optimized through mmWave signals in line-of-sight (LoS) conditions. In non-LoS scenarios, the system switches to μWave mode for obstacle-penetrating signal transmission. Subsequently, we propose a reinforcement learning (RL) approach to optimize cache decisions through the approximation of the Q action-value function. The proposed technique undergoes iterative learning, adapting to dynamic network conditions to enhance the content placement policy and minimize delay. Extensive simulations demonstrate the efficiency of our proposed approach in significantly reducing network delay compared with benchmark schemes.